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Executive Summary 



This paper focuses on transition programs for youth to postsecondary education, broadly 
considered. We address the following questions: (a) What models or programs of transition 
exist? (b) On what basis can we say one transition program is more effective than another? In 
other words, how is successful transition defined? (c) How are transition models and programs 
evaluated? and (d) What is the impact of transition programs, specifically those that aim to 
facilitate transition from one educational system to another, to program completion, or to specific 
career-related employment for disadvantaged youth? 

We identified 16 different general paths that transition programs could potentially 
address and targeted 9 for this systematic review. A literature search of over 8,000 citations 
yielded over 100 studies that warranted further examination. Most paths we identified as 
potential targets for interventions appear not to have been studied using a comparison group 
design, and we were only able to meta-analyze two paths, which we combined, that had 19 
studies of interventions that aim to keep students in college once they get there. 

The 19 studies suggest small but potentially important effects on short-term grades 
earned by program participants. However, several studies employed comparison groups that 
appear to lead to an artificial underestimation of program effects (e.g., by comparing students on 
academic probation to students not on academic probation), and some interventions were 
relatively weak (e.g., adding a journal writing component to an English composition class). 
Studies that employed more comprehensive interventions and that used relatively more 
appropriate comparison groups showed more effective results than did studies that used weaker 
interventions, relatively less appropriate comparison groups, or both. 

Even the best studies included in this review are methodologically suspect, with poor 
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reporting on quality indicators (such as attrition) and an almost exclusive reliance on quasi- 
experimental designs. As such, these studies do not provide a very strong basis for making policy 
recommendations. However, this review suggests that there is reason to be optimistic about the 
potential for relatively comprehensive interventions to help students earn better grades and stay 
in school, at least in the short term. From a public policy perspective, this review points to the 
need for more investment in rigorous studies that investigate, at a finer level of detail, the 
specific aspects of programs that are associated with program success. Rigorous studies are also 
needed that investigate the interaction between programs and student characteristics in order to 
determine what types of programs are most effective for which students. 
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Systematic Reviews of Research: 

Postsecondary Transitions - Identifying Effective Models and Practices 

This paper reports on a systematie review of researeh on transition programs designed to 
help disadvantaged populations move into and through postseeondary edueation. We have 
defined transition as individual movement from pre-eollege edueational systems into and through 
the first two years of postseeondary edueation or into related employment. The purpose of this 
review is to (a) deseribe the various transition interventions that exist around postseeondary 
edueation, (b) assess the inferential strength of the researeh on those intervention programs that 
seek to ease transitions into and through postseeondary edueation and to work, and (e) determine 
the impaet these programs have had on sueeessful student transition. 

The National Center for Edueation Statisties (NCES, 2008) defines postseeondary 
edueation as: 

The provision of a formal instruetional program whose eurrieulum is designed 
primarily for students who are beyond the eompulsory age for high sehool. This 
ineludes programs whose purpose is aeademie, voeational, and eontinuing 
professional edueation, and exeludes avoeational and adult basie edueation 
programs. 

Students plan to partieipate in postseeondary edueation for a variety of reasons. However, the 
great majority of them are probably motivated, at least in part, by the eeonomie returns 
assoeiated with postseeondary edueation. In debates over who should pay for and who benefits 
from investment in postseeondary edueation, most agree that edueation beyond high sehool is 
both a publie and a private good. 

As a publie good, it seems elear that a nation’s eeonomie status depends in part on the 
quality and quantity of postseeondary edueation available (Barton, 2008; Paulsen & St. John, 
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2002). Although a higher level of education is not the only factor involved in earning higher 
wages (Kemple, 2008), there are, on average, advantageous economic returns for increased 
levels of education; Returns to baccalaureate degrees surpass returns to associate degrees, and 
those with an occupational certificate (equivalent to two semesters of full-time study) have 
higher earnings than those with just some college but no degree (Grubb, 2002; Marcotte, Bailey, 
Borkoski, & Kienzl, 2005). Worker groups with more education, including career-related 
education (Kemple, 2008) tend to have higher employment rates (Krolik, 2004; Prince & 

Jenkins, 2005), and an educated citizenry makes fewer demands on state social service resources, 
such as welfare and corrections (National Center for Public Policy and Higher Education 
[NCPPHE], 2004). People with higher levels of income also generate more tax revenue and 
economic activity (Barrow & Rouse, 2005; Barton, 2008). 

Most people consider education to be a bridge to a better life, playing a fundamental role 
in improving the socioeconomic status of individuals, families, and communities from one 
generation to the next. In the United States, colleges and universities offer a sense of “limitless 
possibilities for all” (Trow, 2001, p. 121), a key element of the American dream of equal 
opportunity. Unfortunately, postsecondary education enrollment and completion patterns do not 
reflect this ideal. According to the Organization for Economic Cooperation and Development 
(OECD, 2006, as cited in The National Center for Higher Education Management Systems 
[NCHEMS], 2007), the United States ranks near the bottom of industrialized countries in the 
percentage of 25- to 34-year-olds with an associate’s degree or higher and the percentage of 
entering students who complete a degree program. The likelihood of improving U.S. placement 
in the rankings seems low, given that in 29 states, the 4-year graduation rate for public high 
schools has dropped below 75%; in 10 states, fewer than half of high school graduates enroll in 
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postsecondary education within one year. These facts do not bode well for increasing 
postsecondary enrollment, persistence, and completion (NCHEMS, 2007). 

The majority of positions that pay wages or salaries high enough to support a family — 
and almost two-thirds of all jobs — require skills associated with at least some education beyond 
high school (Carnevale & Derochers, 2003). In fact, although high school grades and behaviors 
are associated with long-term employment and earnings outcomes (Rosenbaum, DeLuca, Miller, 
& Roy, 1999), many researchers would agree with Rosenbaum’s (2001) assertion that high 
school records (i.e., grades, attendance, test scores) have little relationship to employment or 
earnings immediately after high school. Employers place little stock in such records; further, 
recent high school graduates are rarely hired for demanding careers — they often receive entry- 
level jobs instead. Many students thus enroll in postsecondary education (e.g., university, 
community college, or programs designed to lead to qualification for skilled jobs) in an attempt 
to improve their employment prospects. 

Mere postsecondary enrollment is insufficient, however. When postsecondary education 
totals less than a year, earnings increases are negligible (Grubb, 2002). Elnderstanding the 
educational transitions that students must navigate into and through postsecondary systems is 
therefore critical to improving opportunities for all students and for disadvantaged students in 
particular. Dropout can occur at several stages in the process of moving from secondary into and 
through postsecondary education. Eor example, students may not have been adequately prepared 
for the academic requirements of postsecondary education. They may have difficulty balancing 
employment, family, and education commitments; they may believe that they are not suited for 
college; or they may feel out of place and find it difficult to make friends and find social support 
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in the college setting. Transition programs help students succeed in the face of such challenges 
and attain their educational goals. 

This review begins with a description of the theoretical frameworks that address college 
choice and postsecondary student departure processes, followed by a literature review of the 
transition landscape, including the typology adopted for this review. The Methods section 
describes the search strategy used to locate studies and provides specific information on how we 
extracted data, evaluated studies for inclusion, and synthesized the results of the studies that were 
found. In the Discussion section, we highlight the most important findings, discuss the need for 
funding of additional rigorous research on transitions, and suggest characteristics of research that 
are important for framing public policy. 

Theoretical Frameworks 

Researchers and practitioners often use theoretical formulations to better understand how 
their efforts can be more effective. In order to understand and address the challenges related to 
secondary to postsecondary transition, several theoretical perspectives are reviewed here. The 
integration of multiple theoretical perspectives is currently a strong trend in the study of college 
choice and student persistence phenomena. Researchers have drawn from economic (e.g., 
Cabrera, Nora, & Castaneda, 1993; St. John, Paulsen, & Carter, 2005), sociological (e.g., 

Paulsen & St. John, 2002; Perna & Titus, 2005), organizational (e.g.. Bean, 1980, 1983), and 
psychological (e.g., Astin, 1984; Eaton & Bean, 1995) theoretical and conceptual models to 
understand these ill-structured problems. Below we review some of these considerations. 

College Choice Process 

Student college choice is described as a “complex, multi-stage process during which an 
individual develops aspirations to continue formal education beyond high school, followed later 
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by a decision to attend a specific college, university, or institution of advanced vocational 
training” (Hossler, Braxton, & Coopersmith, 1989, p. 234). Students cycle through three stages: 
predisposition, search, and choice (Hossler & Gallagher, 1987). Throughout their primary, 
middle, and high school experiences, students determine whether to attend postsecondary 
education, search for information about institutions and financial aid, narrow the options, and 
then decide on an institution (Hossler et ah, 1989; McDonough, 2004). 

More recent research approaches the college choice process with an integrated conceptual 
model that draws from economics of human capital models and sociological theories of cultural 
and social capital (Perna, 2006). Human capital theory predicts that an individual’s investments 
(in this case, investing time and financial resources in education and training) to increase his or 
her abilities will pay off in improved financial and quality of life status (Becker, 1993). 
Sociological theories of cultural, social, and economic capital recognize that resources vary 
among socioeconomic strata of society, and these resources, or capital, take many forms. 

Cultural capital refers to the system of characteristics that defines an individual’s class status. 
Resources such as language skills, cultural knowledge, and mannerisms derived in part from 
one’s parents are examples of cultural capital (Bourdieu, 1986; Bourdieu & Passeron, 1977). 
Social capital theory focuses on how individuals acquire human, cultural, and other forms of 
capital through their memberships in social networks (Coleman, 1988). For example, through 
their relationships with teachers, advisors, and peers, high school students may or may not learn 
about a wide array of postsecondary options, careers, or financial aid resources. 

Theories of Postsecondary Student Departure 

Considered paradigmatic in stature, Tinto’s Interactionalist theory (1975, 1987, 1993) 
primarily addresses voluntary student departure decisions within postsecondary institutions. 
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Tinto (1975) postulated that students possess various characteristics that directly influence their 
decisions to stay in or leave college. Central to Tinto’s theory is the degree to which a student 
becomes integrated into the academic and social realms of the institution. Academic and social 
integration influence a student’s subsequent commitments to the institution and to the goal of 
college graduation. Finally, Tinto postulates that the greater the levels of institutional 
commitment and commitment to the goal of college graduation, the more likely the individual 
will persist in college. 

Tinto’s (1982, 1986, 1987, 1993) revisions to his theory addressed the importance of 
financial resources within the set of background characteristics with which a student enters a 
postsecondary institution, and acknowledged the role communities external to the institution (e.g., 
family, work, and community) play in students’ departure decisions. Similarly, Bean and Metzner’s 
(1985) model builds on Tinto’s but emphasized the importance of external influences on the 
persistence of nontraditional students, such as those at community colleges. More recently, Braxton, 
Hirschy, and McClendon’s (2004) theory of student departure in commuter institutions gives 
greater importance to the internal campus atmosphere (e.g., academic communities and 
institutional environment) and students’ life circumstances away from campus. 

Literature Review: The Transition Landscape 
Many aspects of transition intervention programs must be considered in any systematic 
review of transition; These include the legislative foundation for government-funded 
interventions, the populations for whom such interventions have been developed, and the types 
of transition programs in existence. We briefly discuss each of these considerations. 

Federal Laws on Transition to Education and Employment 

The United States has a long history of federal legislation providing for investment in the 
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education and training of the American people (e.g., the Morrill Acts of 1862 and 1890 granting 
land to states to create educational institutions; the G.I. Bill of 1944 [also known as the 
Servicemen’s Readjustment Act], and the Higher Education Act of 1965). Such laws have been 
developed in response to economic conditions and forecasts, census and educational attainment 
data, and inequities in opportunity or outcomes. Historically, federal intervention has enabled 
and supported programs financially and sought to maintain some level of public policy 
uniformity across the United States and its territories. 

Several current federal laws address inequities in opportunities for education and 
training; 

• Section 504 of the Rehabilitation Act of 1973 

• The Americans with Disabilities Act of 1990 (ADA) 

• Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (known as the 
Welfare Reform Act) 

• Workforce Investment Act of 1998 (WIA) 

• No Child Left Behind Act of 2001 (NCLB) 

• Individuals with Disabilities Education Improvement Act of 2004 (IDEA) 

• Carl D. Perkins Career and Technical Education Act of 2006 (Perkins IV) 

• Higher Education Opportunity Act of 2008 

These laws provide opportunities to populations that historically have experienced disappointing 
outcomes in education or the labor market, including low-income workers, people with 
disabilities, migrant students, students at risk of dropping out of high school. Native Americans, 
low-skilled adults, and dislocated workers. Such populations are considered disadvantaged in 
various legislative contexts. 
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Current legislation addresses the need to support learners through the transition from their 
eurrent level of education into higher-level education and training and ultimately employment. 
The points at which such transitions are made are key junctures in learner advancement, in part 
because the next step in a trajectory is not always clear to the learner, and disadvantaged students 
often lack the “college knowledge” (Vargas, 2004) to know how to access that information. A 
goal of much federal legislation is to improve successful movement through those transition 
points. 

Despite the recognition of the importance of transition, the term remains undefined by 
much of the relevant legislation. Only IDEA and the vocational rehabilitation portion of WIA — 
both focused on improving opportunities for people with disabilities — included a specific 
definition of transition or transition services. IDEA defines transition services as “a coordinated 
set of activities” focused on facilitating the “movement from school to post-school activities” 
(Sec. 602, parag. 34). Similarly, in WIA, transition services are “a coordinated set of activities 
for a student, designed within an outcome-oriented process, that promotes movement from 
school to post-school activities” (WIA, Sec. 6, parag. 37). In contrast, the term transition is used 
without definition in Perkins IV, NCEB, and the Welfare Reform Act. 

Populations Targeted for Transition Interventions 

Eor purposes of this review, the term disadvantaged student refers to a student who, due 
to educational, economic, cultural, environmental, experiential, or familial circumstances, may 
be less likely to aspire to, enroll in, or succeed in higher education relative to his or her non- 
disadvantaged peers (Revised Code of Washington, 2009). Generally, this includes individuals 
with disabilities, individuals from economically disadvantaged families, individuals who are 
academically low performing, and individuals with limited English proficiency. Some transition 
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interventions target more than one of these groups, and elements of these programs are likely to 
be customized to address the specific concerns, needs, barriers, or preferences of their target 
groups. 

Students with disabilities. Postsecondary enrollment among students with disabilities has 
increased dramatically since the passage of a series of acts aimed at ending discrimination 
against individuals with disabilities. These include Section 504 of the Rehabilitation Act of 1973, 
the ADA of 1990, and the IDEA amendments of 1997 (now supplanted by IDEA 2004; NCES, 
2000). Between 1987 and 2003, the college participation rate of students with disabilities more 
than doubled, rising from 15% to 32% (Newman, 2005). According to the 1995-1996 National 
Postsecondary Student Aid Study (NPSAS: 96), about 6% of undergraduates reported having a 
disability; of those reporting, the percentage by type included learning disability (29%), 
orthopedic impairment (23%), hearing impairment, (16%), and speech impairment (3%). 
Twenty-one percent of the undergraduates who reported a disability stated they had another 
health-related disability (NCES, 2000). The aforementioned study did not include psychiatric 
disabilities as a category, but this category is of growing concern; among college students, 
psychiatric disabilities appear to be even more common than learning disabilities (Sharpe, 
Bruininks, Blacklock, Benson, & Johnson, 2004). Eor example, from Spring 2000 to Spring 
2005, the number of college students who said they had been diagnosed with depression 
increased sharply from 10% to 16% (American College Health Association, 2000, 2006). 

Although access by individuals with disabilities to postsecondary education has 
improved, the news is not all encouraging. Youth with disabilities drop out of high school at 
about twice the rate of their peers without disabilities: In one study, 28% of out-of-school youth 
with disabilities had left high school with no academic credential (Wagner, Newman, Cameto, 
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Garza, & Levine, 2005). The rate of postsecondary attendance of these students is less than half 
that of their nondisabled peers (Murray, Goldstein, Nourse, & Edgar, 2000), though the pattern 
of institutional enrollment differs for students with disabilities. Students in the general 
population are over four times as likely as those with disabilities to attend a four-year institution; 
however, at the community college level, the likelihood of enrollment is similar for students with 
and without disabilities (Wagner et ah, 2005). 

An earlier study found an association between poverty and having a disability and also 
found that students with disabilities coupled with few financial resources were less likely than 
their more financially well-off peers with disabilities to be enrolled in postsecondary education, 
an investment that could offer needed economic rewards over time (Wagner & Blackorby, 1996). 
The authors of this study also noted that categories of disadvantaged status frequently overlap. 

Students from economically disadvantaged families. Kane (2001) reported “persistent 
and widening gaps” in college attendance among students from different family income levels (p. 
2). Comparing postsecondary enrollment (any vocational/technical, 2-year, or 4-year college) 
between 1980 and 1997, the gap in college attendance between students from the highest and 
lowest parental income quartiles increased from 23 to 30 percentage points. Further, during the 
same time period, the earnings differential between high school graduates and college attendees 
more than doubled. Clearly, low-income students who do not attend college experience limited 
economic mobility. 

Students from low-income backgrounds, especially those who attend K-12 schools with 
few academic resources, are often underprepared for the tasks involved in selecting a college, 
such as exploring potential institutions, investigating financial aid options, and understanding the 
application timeline and process (McDonough, 1997, 2004). In a study of youth from working 
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poor backgrounds, McSwain and Davis (2007) noted the laek of guidance students reeeived in 
navigating the college deeision-making proeess, in whieh many felt alone and unsupported in 
their quest for postsecondary education. 

Further, although getting into eollege is one hurdle for students from disadvantaged 
soeioeeonomic baekgrounds, earning a degree is an even greater ehallenge (Dickert-Conlin & 
Rubenstein, 2007). Though students whose families live in or near poverty are often eligible for 
grants and federally subsidized loans, the full level of financial need for tuition and fees is often 
not met (McSwain & Davis, 2007). Students’ fragile financial status is often compounded by a 
conflict between work and sehool, leading many to enroll part time initially and then drop out if 
their employment cireumstances ehange. Enrolling in fewer eourses may affect the amount and 
type of financial aid students are eligible for and is also assoeiated with lower levels of degree 
attainment (Calcagno, Crosta, Bailey, & Jenkins, 2006; McSwain & Davis, 2007). 

Another vulnerable population is youth in foster care. As a group, their rate of high 
school dropout ranges from two to five times higher than youth in the general population. 
Similarly, these young people attend postsecondary education, earn eredits, and eomplete 
degrees at low rates (Berzin, 2008). 

Academically low-performing students. Many students, whether coming directly from 
high school or returning as older learners, fail the eollege placement test and are redireeted to 
remedial edueation. The actual number of students requiring remediation varies with the loeation 
and type of institution (i.e., urban or rural, eommunity eollege or four-year college or university), 
but in 2000, 42% of freshman students in eommunity and teehnical eolleges were enrolled in at 
least one remedial course (Parsad & Lewis, 2003). 

Students can underperform academically for a variety of reasons. For example, they may 
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have relatively low levels of aeademic skills, faee competing demands for their time (such as a 
job or family responsibilities), or experience social or academic adjustment issues. These 
challenges might impede their ability to demonstrate acceptable academic progress. This could 
threaten a student’s ability to maintain good academic standing, retain a scholarship, or graduate 
within a reasonable time frame. 

Students with limited English proficiency. Educational attainment levels differ between 
young adult U.S. citizens and immigrants. In 2005, among students aged 18-24, 86% of U.S. 
citizens graduated from high school compared to 70% of immigrant youth. Although almost 50% 
of native-born students attended at least some college, only about a third of immigrants enrolled 
in postsecondary education (Erisman & Eooney, 2007). Attainment levels vary among 
immigrants by region of origin, age, and generation status. Eatin American and Caribbean 
immigrant educational attainment is generally lower than that of European, African, and Asian 
immigrants. The highest percentage of young adult immigrants (aged 18-24) comes from Eatin 
America; Asian countries send the highest percentages of adult immigrants aged 25-54 
(Rumbaut, 2004, as cited in Erisman & Eooney, 2007). Compared to the first generation 
immigrant, second generation immigrants are more likely to get a high school diploma but are 
less likely to earn an advanced degree (Haskins, 2008). An important consideration in educating 
immigrant populations is to acknowledge that their cultural characteristics vary, implying that 
programs targeting these individuals will likely need to be significantly tailored to their specific 
backgrounds and characteristics. Such attributes may differ by individual, family, language 
status, and neighborhood and schooling environments (Rong & Preissle, 2008). 

A significant barrier for immigrant educational attainment is limited English proficiency. 
Ruiz-de-Velasco and Eix (2000) reported that during the 1993-1994 school year, less than half of 
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English as a Second Language (ESL) public high school students were enrolled in ESL classes or 
bilingual education programs (as cited in Erisman & Looney, 2007). Without instructional 
support to improve their English prohciency, immigrant youths may surrender their educational 
aspirations during high school. Eor example, in the classroom, students with limited English 
proficiency are often not able to keep up with the pace of instruction and some instructors do not 
make any adjustments for students who speak little English (Clark, Hernandez, & Burkey, 2007). 
These students may also lack role models. As a specific example, the Indiana state public school 
system enrollment of limited English-speaking students increased each year between 1992-1993 
(total 4840) and 2006-2007 (total 42,728). During that same time span, the percentage of 
minority teachers in the system declined slightly from 5.2% to 5.0% (Clark et al., 2007, 2007). 
Not surprisingly, the lack of bilingual role models, staff, and counselors pose significant barriers 
to high school and postsecondary attainment. 

Higher education presents a pivotal opportunity for many immigrant students and their 
families. Similar to other disadvantaged students, without sufficient transition support at ah 
education levels, individuals with limited English language skills may resign themselves to low- 
wage jobs and limited social mobility and give up on achieving the American dream (Erisman & 
Looney, 2007). 

Transition Program Typology 

Transition programs can be categorized along a number of dimensions and address 
several domains in a student’s life. One general dimension is whether programs are preparatory, 
supportive, or both. Programs may be academically preparatory in nature, such as those teaching 
content and skills that are valuable in college. Other programs are more supportive, providing 
information about applying for college and financial aid, economic assistance to qualified 
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students, or social services that address the barriers a student may face. Many programs are both 
preparatory and supportive, intervening as necessary to improve transition outcomes. 

Another dimension along which to categorize transition programs is the target audience. 
Target groups include age groups or grades in school as well as a range of populations, described 
below. A third dimension is program content elements. In some cases, an element of program 
content (such as career exploration) is the defining element of an entire program, whereas in 
other cases, the same element is a minor aspect of a wider-ranging program. Other program 
elements, such as dual enrollment, occur with high frequency or exclusively in a particular 
timeframe within the transition sequence. Still others, such as self-advocacy skills, occur 
primarily or only in association with a particular target group. 

Although there are others, the dimension along which we have chosen to categorize 
transition programs for this review is the educational level of the transition. Figure 1 represents 
the many types of transition programs across secondary and postsecondary education levels. 
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Figure 1. Transition program typology. Note. The solid arrows indicate priorities for this review. 



The solid lines in Figure 1 indicate priorities for this review and define its scope. Thus 
the transition paths that were reviewed here are; (a) high school to postsecondary education 
(Paths 1 and 2, see Figure 1); (b) retention and completion of either a community college 
program or the first two years at a 4-year institution (Paths 3 and 4); (c) for out-of-school youth 
only, pre-college and out-of-school programs leading to postsecondary education (Paths 5 and 
6); (d) school (including high school, community/technical college, and university) to career- 
related employment (Paths 7, 8, and 9); and (e) community college to 4-year university (Path 
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10). The dashed lines (Paths 6, 11-16) indicate areas of transition that we considered to be 
outside the scope of this review. 

Many transition programs cluster onto these paths, although the scope of some programs 
might include more than one line. Below, each transition path that was researched for this review 
is described. 

High school to community or technical college (Path 1). Most of the programs that 
work to ease the transition from high school to postsecondary education were designed for the 
transition to a four-year college. However, Tech Prep is a credit-based transition program that is 
more specific to community college. In Tech Prep, high schools and community colleges 
develop articulation agreements so that students need not take technical or occupational content 
in college that they have already learned in high school. Tech Prep li nk s the final two years of 
high school and two years of community college through a sequenced program of study in a CTE 
field, usually leading to an associate degree, often in less time than non-Tech Prep students. 
However, it is a general transition program, not one developed for the at-risk populations that are 
the subject of this paper. 

High school to 4-year college/university (Path 2). Interventions to ease the transition 
from high school to four-year colleges and universities vary greatly, from preparatory options 
such as career academies, dual enrollment, and extracurricular programs, to support interventions 
that provide students with the cultural capital they will need to succeed. 

State government has a role in easing the transition from high school to college. Boswell 
(2000) described 10 policy options that states should consider in this regard, including aligning 
the skills required for high school graduation with college placement requirements, providing 
distance learning options for high school students, developing programs to identify and help at- 
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risk students early in their aeademie eareers, and developing eomputerized information systems 
that allow students to be traeked through the K-12 system, postseeondary edueation, and the 
workforee. 

Dounay (2006) eolleeted high sehool graduation requirements and four-year college 
admissions requirements across all 50 states and concluded that there is often a gap between the 
number of Carnegie units - and the specific courses in those units - required for high school 
graduation and those required for admission to a four-year college or university. Similar 
examinations of smaller subsets of states have yielded the same results (Brown & Niemi, 2007; 
Callan, Finney, Kirst, Usdan, & Venezia, 2006; Hughes & Mechur Karp, 2006), creating a clear 
policy direction for states interested in increasing the numbers of students moving from high 
school to four-year colleges and universities. The National Governors Association (2009a) 
recognized this imperative, but also included other means of aligning secondary and 
postseeondary education, such as increasing student participation in rigorous college preparatory 
courses and holding the K-12 system accountable for student success entering postseeondary 
education. 

Some of the ways in which this alignment is already being developed are through credit- 
based transition programs. These include dual enrollment, middle and early college high schools. 
Advanced Placement, and International Baccalaureate programs (Mechur Karp & Hughes, 

2008), the latter two of which are designed for higher-achieving students and will not be 
reviewed here. Dual enrollment and middle and early college high school programs share certain 
elements with more targeted interventions, such as a focus on improving student achievement 
and preparation for postseeondary education. 
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Interventions aiding the transition from high sehool to four-year postseeondary 
institutions often inelude both aeademie and preparatory components, as seen in familiar 
programs such as AVID (Advancement Via Individual Determination), Upward Bound, or 
GEAR UP (Gaining Early Awareness and Readiness for Undergraduate Programs). Although 
these programs differ from one another in various ways, they also have several elements in 
common, such as the dual focus on academic preparation and social enrichment. Many of these 
programs offer tutoring or study skills, and they often also offer academic enrichment in the form 
of Saturday, after school, or summer sessions. Some of these programs, including Upward 
Bound and GEAR UP, may also offer scholarships. 

Completion of community or technical college (Path 3). The transition through 
postseeondary, especially at the community college level, is a difficult one that many students 
never complete. Tinto’s theory of social and academic integration is often invoked to help 
explain why community college students fail to complete their programs. Because community 
colleges are often commuter colleges, it is harder for students to become committed to the 
institution and therefore complete their degrees (Bean & Metzner, 1985). As such, interventions 
designed to increase institutional engagement might effectively be employed to increase 
graduation rates (Mechur Karp, Hughes, & O’Gara, 2008). Alternative explanations for the 
failure of many community college students to complete programs include (a) unrelated external 
pressures on students such as work or family demands and (b) changes in students’ stated goals 
from initial enrollment (Bailey, Eeinbach, & Jenkins, 2006). 

Given these issues and the fact that for many students, a degree is not their stated goal, it 
is difficult to gauge the effectiveness of community colleges in helping students persist through a 
certificate or degree program. However, there is no shortage of programs and interventions to 
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address this transition. Some eommon practiees inelude (a) student services such as advising, 
counseling, mentoring, and orientation programs, (b) learning communities, in which students go 
through a program in a cohort in order to have more engaging experiences and more 
opportunities for intellectual and social interaction, (c) developmental education and other 
services for academically underprepared students, and (d) college-wide reform, including 
focused attention on research and the use of data to drive program improvement (Bailey & 
Alfonso, 2005). Other recommendations to improve student outcomes include supplemental 
instruction, trained tutors, incorporating subjects such as math or writing instruction into other 
classes, professional development on different learning styles, and emailing personalized interim 
reports to students (Bashford & Slater, 2008). 

Some research has found that students who reach certain milestones (e.g., obtaining 20 
credits or completing 50% of a program) have a higher probability of graduating (Calcagno et 
ah, 2006). These milestones are sometimes called “momentum points.” Guides have been 
developed that allow colleges to use longitudinal student record data to examine momentum 
points in order to learn which college-specific practices are associated with successful student 
outcomes (Leinbach & Jenkins, 2008). 

Community colleges are also accelerating their programs by integrating what was 
previously considered noncredit or prerequisite material into credit-bearing courses so that 
students more quickly gain the knowledge, skills, and credentials required to enter specific 
careers and maintain the motivation to stay in school (Bragg et ah, 2007). The best examples of 
career/employment transition programs integrate academic subject matter with technical skills 
training so that as students continue in the program, they not only gain occupational skills but 
also become more prepared for further education. This type of intervention is usually provided 
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early in a student’s college career and is not seen as necessary to support students’ transition 
from postsecondary education to employment. 

In addition to the institutional interventions listed above, other attempts to improve 
community college student success include developing state policies such as publicizing 
community college enrollment and completion data, targeting funding to specific populations or 
occupations, and more closely overseeing the quality of programs (Palmer, 1998). But 
notwithstanding such practices, Bailey, Calcagno, Jenkins, Kienzl, and Leinbach (2005) noted 
that some of the strongest correlations between institutional factors and student success have 
disturbing implications. Examining a large institution, they found that having relatively large 
percentages of part-time and minority students negatively correlated with successful student 
outcomes. This suggests that colleges wishing to improve their completion statistics might do so 
by inappropriately limiting access to some groups of students. Bailey et al. recommended further 
research to identify those community college characteristics and policies that promote student 
success. 

Completion of 4-year college/university (Path 4). This transition, especially the 
completion of the freshman year, has been abundantly studied in recent years because many 
students enter colleges and universities, but just a fraction actually complete a degree. Only 43% 
of students from the class of 1992 completed a postsecondary degree within eight years of 
graduating high school (National Center for Education Statistics, 2006, Table 306). According to 
these data, another 30% of Class of 1992 students began college but did not complete a degree 
program. Clearly student persistence is an issue, and the transition from high school to college 
extends into the first years of the four-year college experience. 
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As described above, the most-cited theoretical framework for college integration, 
persistence, and success was laid out by Tinto (1975, 1987, 1993). This model has become the 
starting point for deeper examinations of the factors involved in student success. For example, 
Robbins et al. (2004) suggested that psychosocial factors (such as academic self-efficacy) play a 
large role in determining whether students will persist beyond the first year. Building on this 
work, the Educational Policy Institute (Texas Guaranteed Student Loan Corporation and the 
Educational Policy Institute, 2008) offered specific strategies to increase student retention and 
success. Their book outlined specifics on how faculty and staff can help support student 
engagement on campus and delineated three types of advising that are important for student 
success: (a) academic advising, which may direct students to first-year success programs, 
tutoring, and study skills or time management courses, (b) financial advising, which can help 
students grapple with the high cost of education — including tuition, fees, books, and living 
expenses — without accumulating excessive debt, and (c) career counseling, which may fill 
students’ need for information about desired careers and what it takes to succeed in them. 

In addition to these kinds of advising, colleges and universities are addressing social or 
personal barriers to success. Eor instance, many students must continue to work while attending 
school in order to support their families. This extends the time period of commitment to 
education, which negatively affects the likelihood of completion. Students may be the first in 
their families to go to college, so they may need orientation and integration support. They may 
have cultural, disability, or health issues that present barriers to success. All of these issues may 
be addressed through various kinds of support services including counseling, mentoring, or 
assistance with transportation. 
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Pre-college education to community or technical college (Path 5). As pressing as the 
needs are for seeondary students to meet postsecondary admissions requirements, adults make up 
a large part of the underprepared population. They range from native-born Americans who did 
not complete high school to immigrants who need to learn English before they can transition to 
higher education or employment. The transition stages for adults are often multiple: from adult 
basic education (ABE) or English as a Second Eanguage (ESE) through the General Educational 
Development (GED), often to community college developmental education, and finally to 
college-level coursework. There are many steps along the way during which adults may “stop 
ouf ’ or take a break from enrollment. 

In order to help adults make these transitions faster, adult career pathways have been 
developed by various organizations and institutions (Bragg et ah, 2007). The Center for 
Occupational Research and Development (CORD) defined an adult career pathway as consisting 
of “the guidance, remediation, curricula, and other support elements required to enable career- 
limited adults to enter the workforce and progress in rewarding careers” (Hull & Hinckley, 2007, 
p. vii). The elements of adult career pathways listed by Hull and Hinckley are similar to those in 
secondary-level transition programs, only developed for adults rather than adolescents. Eor 
example, common support elements for adults transitioning into postsecondary education include 
child care and transportation aid. With respect to curriculum, adult career pathways are often 
designed to integrate remediation with introductory career preparation in order to accelerate 
adults’ completion of certificate and training programs (Prince & Jenkins, 2005). 

High school to related employment (Path 7). Career and technical education (CTE) is 
one program area in high school that can ease the transition from school directly to work for 
students who do not have immediate plans to attend college. An economic study showed that the 
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returns to CTE coursework (in terms of labor market payoffs) for students who entered work 
immediately after high school were higher than for academic coursework (Mane, 1999). This 
advantage held true for both the short and medium term (i.e., up to seven years after graduation). 
The study did not extend past that period, but the author noted that although the returns to high 
school CTE were substantial, in general the returns to postsecondary education were higher still. 

College/university to related employment (Paths 8 and 9). College placement services 
represent the largest type of interventions to assist students in their transition from community 
college to related employment. One element of many programs that relates to this transition is an 
emphasis on work readiness or SCANS skills (Secretary’s Commission on Achieving Necessary 
Skills, 1991). Skills such as working in a team, employing technology, and being punctual are 
commonly a part of transition programs aimed at helping graduating students find employment. 

States need educated workforces in order to compete in the national and global 
economies. State governors are creating workforce development systems that engage businesses 
and provide seamless transitions across educational systems, as described here. The National 
Governors Association (2009b) has as its goal the development of state workforce development 
systems that (a) are flexible and responsive to changing state needs and (b) offer residents the 
opportunity for lifelong learning, entering and re-entering educational systems as their lives 
dictate. As an example. Prince and Jenkins (2005) showed the benefit of successfully crossing 
transition points: Attending college for at least one year and earning a credential provides a 
substantial boost in earnings for adults who begin with a high school diploma or less. 

Community or technical college to 4-year college/university (Path 10). Intervention 
programs for this transition include community college “Eirst Year Experience” programs that 
community colleges have implemented in order to increase student integration into the 
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postsecondary experience. These programs include preparatory as well as supportive elements. 
Learning communities are a common feature of these interventions, following the belief that a 
block schedule with the same students allows community college students to quickly find study 
partners and make new friendships (Bloom & Sommo, 2005; Tinto, 1997), thus promoting 
student integration and achievement. 

Research Questions 

We searched for empirical studies on the above-described transition paths in order to see 
what lessons can be learned from a synthesis of these studies. Specifically, we set out to conduct 
a systematic review and, if appropriate, a meta-analysis (or several meta-analyses) of studies that 
attempt to determine the effectiveness of transition programs. In the sections that follow, we 
detail (a) the methods of the systematic review, (b) the results of the one meta-analysis we were 
able to conduct, (c) the methods and results of studies that met our inclusion criteria but could 
not be included in the meta-analysis, (d) the limitations of the meta-analysis, and (e) suggestions 
for both policymakers and future research. 

Methods 

Inclusion Criteria 

To be included in this review, a study had to meet several criteria. As described earlier, 
we examined federal legislation for labels that might be used to describe individuals who, from a 
public policy perspective, might be intended populations for targeted transition services. As 
such, our first criterion was that the sample had to be composed primarily of students who lit this 
broad definition of disadvantage (see Table 1 for the terms related to disadvantage that were used 
in the electronic literature searches). 
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Table 1 

Literature Search Keywords for Postsecondary Transitions 



Terms to suggest a 
transition 


Terms to suggest an 
empirical study 


Terms to suggest an 
intervention 


Terms to suggest 
disadvantage 


Transition* 

Career 

School to work 

College 

Employment 

Articulation 

Vocation* 


Outcome 

Results 

Compar* 

Empirical 

Effect* 

Study 

Experiment* 

Eongitudinal 


Program* 

Intervention* 

Enrichment 


At-risk 

English Language 
Learners 

EEE or English language 
learner 

English learner 
ESE or English as second 
language or English 
as a second 
language 

EEE or English as 

foreign language or 
English as a foreign 
language 

EEP or limited English 
proficien* 
Multilingual 
Immigrant 
Bilingual 
Foreign-bom 
Second language 

acquisition or SEA 
Non-English speaker 

Students with 

Disabilities 

Disab* 

Individualized Education 
Plan or lEP 
Rehabilitat* 

Section 504 
Chronic health 
Chronic disease 

Children of Migrant or 

Seasonal Workers 

Migrant 

Seasonal 

Migratory 

Economically 
Disadvantaged Students 
Disadvantage 
Homeless 
Eow income 
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Terms to suggest a 
transition 


Terms to suggest an 
empirical study 


Terms to suggest an 
intervention 


Terms to suggest 
disadvantage 








Public assistance 

Low socio-economic 

status or low SES 

Poor 

Welfare 

Poverty 

Students in the Juvenile 
Justice System 
Offender 
Delinquen* 

Correctional or 
corrections 
Probation 
Justice 

Students Who Are Low 

Achieving 

Basic skills 

Below grade level 

Low achiev* 

Underachieving 

Retained 

Fail 

Held back 
Remedial 
Dropout 
Underprepared 



Notes. Terms were connected with OR within columns, and with AND between columns, so that to be identified through the 
electronic search, the database had to index at least one term from each column. Terms followed by an * are truncated; the search 
identified any term that includes the letters before the truncation symbol. For example, disab* identifies “disability,” “disabled,” 
and “disabilities,” as well as “disabuse.” 



In addition, a study had to describe a formal program or intervention that addresses a 
transition from secondary education to college or university, or from a secondary education to 
related employment. As such, this review focuses on specific interventions (such as those 
designed to facilitate the transition to college), and larger transition programs (such as GEAR UP 
and career academies), but not transition policies (such as credit transfer policies). Further, by 
focusing on programs that help students transition from secondary education to college, through 
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college, or from seeondary edueation to related employment, we exeluded those programs that 
help with the transition from eollege to related employment, or from seeondary edueation to the 
general work foree (i.e., programs that teaeh soft employment skills but do not train partieipants 
for a eareer, of. Figure 1). 

We also required that inoluded studies measure an outoome olosely related to the goals of 
the transition intervention. For example, assuming it met other oriteria, we would have inoluded 
a GEAR UP evaluation that assessed student enrollment in postseoondary edueation or eollege 
readiness among high sohool students. We would have not inoluded a GEAR UP evaluation that 
only measured the program’s effeots on aoademio aohievement in middle school, as this outcome 
is too distal to interpret with a reasonable degree of oonfidenoe. 

Einally, to be inoluded in this review, a study needed to provide a quantitative evaluation 
of the effeots of a transition program. Ideally, the evaluation would involve random assignment 
to oonditions in the oontext of an experiment (e.g., of students to either a transition program or to 
a wait-list oontrol), but we antioipated that these would be rare; as suoh, we required only that the 
study inolude a looal oomparison group. See Quality assessment of included studies for details of 
how we further assessed the quality of studies. 

Literature Search 

We used several strategies to find relevant literature, as reoommended in the literature on 
systematio reviewing (see Rothstein, Turner, & Eavenberg, 2004). Eirst, we eondueted searehes 
of the eleetronie databases PsyeINFO and ERIC. The seareh terms we used are given in Table 1. 
The titles and abstraets from eitations identified via the database seareh were examined by at 
least two individuals working independently. We obtained full eopies of a study if, after 
diseussion, both individuals agreed that it might meet our inelusion oriteria. 
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In addition, we identified 124 documents that could, broadly speaking, be conceptualized 



as literature reviews of transition programs. For each of these reviews, we examined the 
reference sections for citations that appeared to meet our inclusion criteria and attempted to 
obtain copies of these studies if they did. 

Finally, we conducted searches of the websites of foundations, research organizations, 
and governmental agencies; in all, over 70 were searched (see Table 2). Based on our sense that 
research on this topic often does not appear in peer-reviewed outlets, we believed that this was a 
critical step in locating relevant studies. 

Table 2 

Websites Searched for Possibly Relevant Studies 



Organization 


Internet address 


Academic Pathways to Access Student 
Success 


www.anass.uiuc.edu 


Academy for Educational Development 


www.aed.ors 


Achieve, Inc. 


www.achieve.ors 


Association for Career and Technical 
Education Research 


w ww. asr i. wsu. edu/ acter/ 


American Association for Community 
Colleges 


w ww. aacc .nche . edu 


American Institutes for Research 


www.air.ors 


American Youth Policy Forum 


www.avnfors 


Annie E. Casey Foundation 


www.aecfors 


Association for Supervision and Curriculum 
Development 


www.ascd.ors 


Association for Career and Technical 
Education 


www.acteonline.ors 


Association of American Colleges and 
Universities 


www.aacu.ors 


Berea College 


www.berea.edu 


Broad Foundation 


www.broadfoundation.ors 


Carnegie Foundation 


www.camesiefoundation.ors 


Career Education Corporation 


www.careered.com 


The Center for Community College Policy 


www.communitvcollesenolicv.ors 


The Center for Educational Policy Research 


www.s4s.ors 


Center for Occupational Research and 
Development 


www.cord.ors 


The Center for Research on Developmental 
Education and Urban Eiteracy 


www.cehd.umn.edu/crdeul/about.html 


Center on Education and Work 


www.cew.wisc.edu/ 
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Organization 


Internet address 


College of the Ozarks 


www.cofo.edu 


Community College Resource Center 


www.ccrc.tc.columbia.edu/ 


Council for Exceptional Children 


www.cec.sDed.ors//AM/TemDlate.cfm?section=Home 


ED Publications (U.S. Department of 
Education) 


www.ed.eov/about/nubs/intro/index.html?src=au 


Education Commission of the States 


www.ecs.ora 


Educational Policy Institute 


www.educationalpolicv.ora 


The Education Trust 


w ww2 . edtrust. or a/edtrust/ 


Ford Foundation 


www.fordfound.ora 


Bill and Melinda Gates Foundation 


www.aatesfoundation.ora/Paaes/home.asnx 


The Institute on Education and the 
Economy 


www.tc.columbia.edu/centers/iee/ 


Jobs for the Future 


www.iff.ora 


W.K. Kellogg Foundation 


www.wkkfora 


Eatin America Research and Service 
Agency 


www.larasa.ora 


Eeague for Innovation in the community 
College 


www.leaaue.ora/index.cfm 


Eeague of United Eatin American Citizens 


www.lulac.ora 


Filly Foundation 


www.lillv.com/nroducts/access/foundation.html 


Eumina Foundation 


w ww. luminafoundation. ora 


Ronald E. McNair Postbaccalaureate 
Achievement Program (TRIO) 


www.ed.aov/nroarams/triomcnair/index.html 


MDRC (Manpower Demonstration 
Research Corporation) 


www.mdrc.ora 


Mathematica Policy Research, Inc. 


www.mathematica-mnr. com/ 


MEEMAC Education Foundation 


www.melmacfoundation.ora 


Charles Stewart Mott Foundation 


www.mott.ora 


MPR Associates 


www.mnrinc.com 


National Association for the Advancement 
of Colored People 


www.naacn.ora/home/index.htm 


National Career Pathways Network 


www.cord.ora/ncnn-index.cfm 


National Center for the First-Year 
Experience and Students in Transition 


www.sc.edu/fve/ 


National Center for Public Policy and 
Higher Education 


www.hiahereducation.ora/index.shtml 


National Council for Workforce Education 


www.ncwe.ora 


National Governors Association 


www.naa.ora 


National Research Center for Career and 
Technical Education 


www.nrccte.ora 


National Center for Research in Vocational 
Education 


vocserve.berkelev.edu 


Nellie Mae Education Foundation 


www.nmefdn.ora 


National Science Foundation 


www.nsf aov 


Office of Community College Research and 
Eeadership 


occrl.ed.uiuc.edu/ 


Office of Special Education Programs 


www.ed.aov/about/offices/list/osers/osen/index.html 


Office of Vocational and Adult Education 


www.ed.aov/about/offices/list/ovae/index.html 
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Organization 


Internet address 


Frederick D. Patterson Institute 


httD://www.natterson-uncf.or2/index.htm 


Pew Charitable Trust 


www.newtrusts.0r2 


Postsecondary Education Opportunity 


www.nostsecondarv.or2 


Rockefeller Foundation 


www.rockfound.or2 


United Negro College Fund 


www.uncf.0r2 


Upjohn Institute for Employment Research 


www.uni0hninst.0r2 


Washington Center for Improving the 
Quality of Undergrad Education 


www.ever 2 reen.edu/washcenter/nroiect.asn?nid =73 


Washington State Board for Community 
and Tech Colleges 


www.sbctc.ctc.edu 


WestEd 


www.wested.or2/cs/we/nrint/docs/we/home.htm 


Women Employed 


www.womenemnloved.or2 


Workforce Strategy 


www.workforcestrate2v.or2/ 



After obtaining electronic or physical copies of the studies that passed through our initial 
screen, we utilized a secondary screening process to determine if studies actually met our 
inclusion criteria. As such, studies that looked promising based on a reading of their titles and 
abstracts were evaluated for inclusion using the full study text, as is often recommended (see 
Cooper, 1998). See Table 1 in the Appendix for the instrument we used to guide this process. 
Coding Studies 

After the second screening, all studies that seemed to meet our inclusion criteria were 
fully coded. As can be seen in Table 2 of the Appendix, we coded background study 
characteristics (e.g., authors, year of publication), characteristics of the intervention (e.g., the 
type of transition addressed, the duration of the intervention), characteristics of the sample (e.g., 
age, source of disadvantage), and outcomes (e.g., construct measured, effect size). All studies 
were then coded using the protocol given in Table 3 in the Appendix as a guide. 

Quality assessment of included studies. We attempted to assess, as part of the coding, 
the likely internal, external, construct, and statistical validity of the inferences arising from all 
studies. We approached this aspect of the review using the framework provided by Valentine and 
Cooper (2008). This approach attempts to overcome many of the shortcomings of existing 
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quality scales. Among these are the reliance on single scores to represent the multidimensional 
eonstruet of study quality and the use of items that require a great deal of inferenee on the part of 
the study eoders. Speeifieally, we coded questions that addressed the internal validity (e.g., how 
partieipants were assigned to eonditions, overall attrition rate), external validity (e.g., degree to 
whieh the sample appeared to be representative of the target population), eonstruet validity (e.g., 
reliability of scores), and statistieal validity (e.g., the extent to whieh data met assumptions 
underlying the general linear model). As will be shown later, however, due to poor reporting in 
the primary studies, we were unable to assess studies on most quality dimensions. 

Data Analysis 

To analyze data, we conducted a me ta- analysis treating eaeh independent sample within 
the studies as the unit of analysis (most studies provided only one independent sample). We 
weighted effeet sizes by the amount of information they provided about the population mean 
(i.e., we used the typical inverse varianee weight) and ealeulated 95% confidenee intervals for all 
effeets. In addition, this data analysis strategy required us to make several decisions about how 
we would address certain complexities in the data. Each of these is described below. 

Effect size metric. We employed effeet sizes that reflect mean differenees between 
groups for continuous outcomes. In addition, beeause we antieipated that most studies would 
ehoose to operationalize their construets in different ways, we standardized these effect sizes to 
provide an interpretable eomparison across studies. The standardized mean difference effect size 
is computed as: 



where is the treatment group mean, is the eomparison group mean, and Sp is the pooled 
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standard deviation. This formulation of the standardized mean difference effect size has a known 



bias in small samples, so we applied the usual correction for this (Hedges, 1981). Standardized 
mean difference effect sizes were computed so that values greater than zero indicated positive 
program effects (e.g., better grades for the intervention group relative to the comparison group). 

For dichotomous outcomes (e.g., persisted vs. did not persist), we employed the odds 
ratio, which is defined as 

a 

OR = — = ad ^bc 
c 

d 

where a is defined as the number of successes in the intervention group, b is the number of non- 
successes in the intervention group, c is the number of successes in the comparison group, and d 
is the number of non-successes in the comparison group. Meta-analysis was performed on logged 
odds ratios, because this metric has better statistical properties (Lipsey & Wilson, 2001). We 
then transformed the logged odds ratios to odds ratios for presentation. The odds ratios were set 
up so that values greater than one indicated positive program effects (e.g., greater persistence 
rates among intervention group members relative to comparison group members). 

Dependencies among effect sizes. Independence of observations is a fundamental 
assumption underlying the general linear model, but in a systematic review, effect sizes 
(observations) can be dependent for several different reasons. For example, researchers may 
assess an outcome at the end of a program and at a subsequent follow-up. Or, researchers might 
collect multiple measures of the same construct (e.g., scores on a standardized test and class 
grades as a measure of academic achievement). Because these effects are based on the same 
sample, they are not independent and treating them as such has undesirable effects. Chief among 
these are that studies will not be weighted properly in a meta-analysis, and statistics that estimate 
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variability (or are based on estimates of variability) will be biased. 

To address the problem of dependence, researchers have choices. One is to randomly 
select one measure to represent the study. This has the virtue of dealing with the dependence 
problem but the drawback of discarding information. Another strategy is to select the outcome 
that maximizes similarity with other studies. Often this strategy is better than randomly selecting 
one effect size, but it still results in a loss of information. A final strategy is often referred to as 
the shifting unit of analysis approach (Cooper, 1998); this is the approach we adopted for this 
paper. The shifting unit of analysis approach involves averaging effects when appropriate (e.g., 
averaging posttest and follow-up effects when testing the overall effect size for the intervention), 
then splitting the effects when testing the dimension on which they differ. In this example, when 
asking whether program effects appear to persist over time, a study with both a posttest effect 
size and a follow-up effect size would contribute to both levels of that moderator. 

Error model. Finally, reviewers need to consider whether to employ a fixed effects or a 
random effects analytic model. Using the fixed effects model, study effects can be thought of as 
being randomly sampled from a single population of studies, and therefore any differences in 
effect sizes across studies are treated as solely due to random sampling and identifiable 
covariates. Using the random effects model, reviewers assume that studies do not in fact share a 
single population value, and any differences in between-study effect sizes are due to random 
sampling error, any identifiable covariates, and other random factors that cannot be identified. 

The choice between fixed and random effects models can be an important one, because 
the confidence intervals arising from a random effects analysis will never be smaller and are 
often larger than their fixed effects counterparts; this has implications for both the statistical 
significance tests and interpreting the likely range of an intervention’s effect. In practice. 
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choosing between the models is done empirically or conceptually. Empirieally, reviewers often 
allow a statistieal test of homogeneity to dietate their choice. The formula for this test is 

^ w,{d-df 
^ k-l ’ 

where J, is eaeh individual effeet size, w; is the inverse variance weight for effeet size i, and d is 
the weighted average effect size. Formally, the homogeneity test is a test of the between-studies 
variance component, and a significant value of Q indieates that the variation among studies is 
significantly different from zero, and often leads reviewers to employ a random effeets model. 

Generally, however, the choice between models is best made eoneeptually (Hedges & 
Vevea, 1998; Valentine, Pigott, & Rothstein, in press). Reviewers might, for example, consider 
the diversity of research designs, programs, outeomes, and samples in their review and use a 
fixed effeets model if these seem to be very similar. Or, they may have an explicit interest in the 
extent to which different studies yield different answers. In addition, reviewers could consider 
their desired universe of generalization as a basis for ehoosing between the models. The fixed 
effects model allows for generalization to studies that are highly like the ones in the review, 
whereas the random effects model allows for inferences that are not so tightly eonditioned on the 
observed studies (henee, the inferenees are more broadly generalizable). In this review, the 
programs meta-analyzed were not highly similar; in addition, we were interested in the extent to 
whieh the studies seemed to yield different effects, and we also wanted to take advantage of the 
broader range of generalization. As sueh, we adopted the random effects data analytic model. 

Results 

Literature Search 

Because we cast a very wide net with our electronic search, we identified over 8,000 
possible citations. The vast majority of these were not relevant to the purposes of our study (e.g.. 
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were opinion pieees, literature reviews, or simple deseriptions of transition programs), and only 
109 were selected for further evaluation. Of these, 33 were selected for full coding as possibly 
meeting inclusion criteria; on further examination, some of these turned out not to meet our 
inclusion criteria. The search of the literature reviews yielded an additional 149 potential studies, 
and the website search yielded an additional 30. As with the electronic literature search, most of 
these could not be included in this review. 

By far, most of the eligible studies were of programs designed to help with the transition 
to college during the first two years of the undergraduate experience (Paths 3 and 4 in Figure 1). 
In fact, of the 9 transition paths identified as priorities for this review as outlined in Figure 1, 
these were the only transitions that have been sufficiently studied to perform a meta-analysis. A 
total of 19 unique studies were identified that studied interventions aimed at helping students 
remain in college. Both community college and four-year institutions were represented, but the 
latter by far represented the largest category. In the analyses below, we combined these different 
institutions because the approaches used were similar. Eighteen of these studies also contained 
enough information to compute an effect size that described the impact the intervention had on 
program participants. 

Description of Interventions 

As anticipated, the interventions included in this review varied in interesting ways. 
Because the interventions were designed to help college students stay in school (i.e., are college 
transition programs), they all included students who were either at increased risk for college 
failure (e.g., were identified as high-risk admits) or were on academic probation. Due to their 
common purpose, the studies were similar in that they included students with a variety of 
background characteristics (e.g., ethnicity) and included both men and women. However, the 
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specific approaches taken in these studies varied quite a bit. These ranged from relatively 
comprehensive interventions (e.g., a seminar designed to facilitate college adjustment, coupled 
with limitations on the number of credit hours students could enroll in, smaller classes, and 
tutoring; Hecker, 1995) to those that were much smaller in scale, such as adding a journaling 
component to an English composition class (Cohen Goodman, 1998). Most interventions fell 
between these two poles, with a freshman orientation/adjustment seminar being the strategy most 
often adopted (either alone or in conjunction with other activities such as tutoring). 

Description of Research Designs and Study Implementation 

Unfortunately, only two of the 19 studies included in this review used a random 
mechanism to assign students to groups. As such, selection (i.e., differences in participants who 
receive the intervention relative to those who do not) is a pervasive threat to the validity of these 
studies. Notably, one additional study (Moss & Yeaton, 2006) employed a regression- 
discontinuity design to study the effects of a developmental English program for college 
students. This study was not included in the meta-analysis due to the difficulties of converting 
the statistical results of the regression-discontinuity design to a metric compatible with the rest of 
the studies in this review, but it will be discussed below. 

Potential selection effects were addressed to varying degrees in the studies included in 
the meta-analysis. Most studies compared the students receiving the intervention to students who 
appeared to be comparable (e.g., met study inclusion criteria) but did not receive the 
intervention. The best of these studies attempted to adjust for baseline differences between 
groups or allowed us to accomplish the same goal. Eor example, some studies computed means 
that were adjusted for scores on college entrance exams, whereas others allowed us to compute a 
pretest effect size that was subtracted from the reported posttest effect size. However, some 
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studies compared program students to students who were clearly not comparable (e.g., Cone, 
1991). We explore the consequences of these design choices below. 

A pervasive threat to the statistical conclusions reached in the individual studies, and 
relevant to the confidence intervals we generated through meta-analyses of them, is that many of 
the included studies likely violated the statistical assumption of independence of observations. 
For example, many studies involved comparing students in sections of a course to students in 
other sections of a course. Because students were not randomly assigned to sections, they may 
share characteristics that increase the similarity of observations within sections relative to the 
similarity of observations across sections. Further, students in the same section share other 
influences, such as the instructor, that likely also tend to increase their relative similarity. Most 
important for the purposes of this review, violation of the assumption of independence can lead 
to standard errors that are spuriously small — and hence, confidence intervals that are too narrow, 
and statistical tests that are too likely to reject the null hypothesis. Unfortunately, the studies in 
this review usually did not give us enough information about the nature and extent of potential 
data clustering, and none allowed us to estimate a likely value that could be used to arrive at 
potentially better standard errors. 

Almost without exception, the studies gave little indication about other potential threats 
to their validity. For example, most studies did not give an indication of attrition (either overall 
or differential), data exclusions (such as systematically missing data), or intervention fidelity. 

The absence of good reporting about these issues means that we do not know how serious these 
threats are to the validity of the conclusions we draw below. 

Outcomes Measured 

Most studies measured academic achievement (usually via grade point average [GPA]) or 
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persistence (i.e., re-enrollment). The majority of studies measured these outcomes either 
immediately after the program (e.g., for a Fall 2000 course, re-enrollment for the Spring 2001 
semester), or one semester later. Only two studies (Clark & Halpem, 1993; Stovall, 1999) can be 
considered to have measured outcomes over the long term (i.e., more than one year post- 
program). 

Data Dependencies 

Abadie (1999) reported outcomes for two cohorts of students. We collapsed these into 
one group for analysis purposes. Abadie (1999) and Stovall (1999) also reported outcomes for 
multiple points in time (Abadie, immediately following the end of the intervention and one 
semester later; Stovall, at the end of the first and second semesters, the end of the second year, 
and the end of the third year). We also collapsed these for analysis. We would have employed 
Cooper’s (1998) shifting unit approach here if more studies had measured outcomes at similar 
follow-up periods, but the lack of the feature across studies made such an analysis impossible. 
McGregor (2001) administered two measures related to academic achievement, and these were 
averaged for analysis. Finally, two studies (Fry, 2007; Hecker; 1995) used two comparison 
groups (one comparison group that was made up of regularly admitted students, and a second 
group that was made up of students who were more like the students receiving the intervention). 
Here, we did employ Cooper’s shifting unit of analysis approach and averaged these groups for 
the overall analysis, but allowed them to contribute to both levels of analysis when we examined 
the nature of the comparison group and its effects on effect size estimation. 

Data Analysis 

Academic achievement. Eighteen studies measured the impact of an intervention on 
academic achievement (most often, GPA; see Table 4 in the Appendix). The random effects 
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estimate was positive, indieating that program partieipants fared better on aehievement related 
outcomes, but not statistically significant, d = 0.08 ± A7,p= .30. The distribution of effect sizes 
was heterogeneous, 2(17) = 68.7, f = 15%, p < .001. 

Persistence. Eleven studies measured the impact of an intervention on student persistence 
(see Table 5 in the Appendix). The random effects estimate for this outcome was positive, 
indicating that program participants were more likely to re-enroll, but not statistically significant, 
with an odds ratio of 1.46 (the 95% confidence interval ranged from .85 to 2.51), p = .17. Once 
again, the distribution of effect sizes was heterogeneous, 2(10) = 84.7, 7 = 88%, p < .001. 

Publication bias. Publication bias — the tendency for studies lacking statistically 
significant outcomes to go unpublished — is a concern in every review, even those that, like ours, 
include a vigorous search for unpublished literature. For the academic achievement outcomes, 
we conducted a statistical analysis to help assess whether our set of studies appeared to be 
affected by publication bias. It should be noted that there are no very good solutions to the 
problems posed by publication bias, and current statistical approaches are at best informed 
guesses about the nature and severity of potential publication bias. We used the trim and fill 
approach, which is based on the assumption that the observed studies (i.e., those in the meta- 
analysis) are a random sample from a normally distributed population of studies. 

Our analysis suggests that, in fact, some degree of publication bias might exist in our set 
of achievement outcomes. Specifically, the trim and fill analysis identified that an additional two 
studies would need to be added to the distribution of effect sizes in the achievement meta- 
analysis for that distribution to be essentially normal. However, adding these studies does not 
substantively alter the interpretation of the overall meta-analysis on achievement outcomes. 
Specifically, the new estimated effect size is still small and not statistically significant, d = 0.01, 
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p = .98, and is neither statistieally signifieantly or praetieally signifieantly different from our 
overall estimate. As sueh, the hints of publieation bias that exist do not seem to suggest a great 
deal of eoneern about the integrity of our meta-analytie dataset. 

Comparison quality as a moderator of study effects. For both grades and persistenee, the 
distributions of effect sizes were heterogeneous. Heterogeneity is one justification for the search 
for moderating influences, as these might explain some or potentially all of the “excess” 
observed heterogeneity. We noted that studies varied in terms of the quality of the comparison 
group against whom the relative effects of the intervention were judged. We therefore 
investigated whether the quality of the comparison group moderated the effect sizes we 
observed. This was very much the case. The five comparisons of the academic achievement of 
program students to clearly non-comparable students yielded strong, negative, statistically 
significant program effects {d = -.45 ± .17, p < .001), whereas the 15 comparisons of program 
students to relatively more comparable students yielded a statistically significant positive effect 
(J = .25 ± .1 l,p < .001). A fixed effects moderator test for the difference between these two 
groups of effect sizes was statistically significant, 2(1) = 68. \,p< .001. Notably, studies within 
each level of comparability appeared to be relatively homogeneous (both p’s greater than .16) 

For persistence outcomes, the nature of the comparison group again moderated the 
observed effect sizes. The four studies that compared the persistence of program students to 
clearly non-comparable students yielded negative effects (the weighted average odds ratio was 
.69, with a 95% confidence interval ranging from .34 to 1.40), whereas the 10 comparisons of 
program students to relatively more comparable students yielded positive effects (the weighted 
average odds ratio was 1.21, with a 95% confidence interval ranging from 1.001 to 1.46). The 
fixed effects moderator test for the difference between these two groups of effect sizes was 
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statistically significant, Q{\) = 24. 4,p< .001. Studies within each level of eomparability again 
appeared to be relatively homogeneous (both p’s greater than .29). 

“Best practice” studies relative to other studies. A final eomparison of interest involves 
those studies that, eompared to the other studies in our review, can be eonsidered “best praetice.” 
That is, these studies used both a relatively intensive intervention and a relatively better 
eomparison group. As expeeted, the nine eomparisons with these eharaeteristies yielded positive 
and statistieally significant effect sizes for aeademie aehievement {d= .29 ± . 15, p < .001), 
whereas the 1 1 eomparisons that laeked either an intensive intervention, a relatively good 
eomparison, or both yielded negative effect sizes in the random effeets model (J = -.17 ± .21, p 
=.12). The fixed effeets moderator test for the differenee between these two groups of effeet 
sizes was statistieally signifieant, Q{\) = 44.5 ,P< .001. Studies within the “best praetiee” 
eategory appeared to be relatively homogeneous (p = .24, f = 23%), whereas studies within the 
lower quality eategory were still heterogeneous (p < .001, 7 = 70%). There was not enough 
variation on the quality dimension to do a similar analysis for studies that measured persistence, 
although an examination of Table 5 in the Appendix suggests that the pattern is similar. 

Interpreting the program effects. If we assume that the best estimate of the effects of 
transition programs on student aehievement eomes from the “best praetiee” studies, this suggests 
that these programs have a population effeet of about 5 = .29 on student grades. To put this in 
eontext, assume that students are expeeted to have a GPA of 2.0 if they do not reeeive the 
intervention. A population effeet of 5 = .29 implies that students reeeiving the intervention 
should be three-tenths of a standard deviation higher in GPA than those not reeeiving the 
intervention. A typieal standard deviation for GPA in the lower portion of the distribution is 
about .75, so the typieal student reeeiving the intervention should have a GPA of about 2.22 [i.e.. 
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2.0 + (.75 X .29)]. For every five program students taking 12 eredit hours, this would be 
approximately equivalent to four of them earning three C’s and one B, with the fifth earning four 
C’s, whereas the five students in the eomparison eondition would earn all C grades. 

One problem with the interpretation of the program effeets for grades is that some 
interventions required students to attend a seminar or eourse, and it was not always elear if or 
how these seminars were graded. It was also not always elear how many credit hours students 
would have enrolled for, if these were formal courses. If the seminars were graded and this grade 
was included in the computation of the GPA, and if students enrolled for three credit hours, then 
the program effect could largely or entirely be due to the influence of the program course on 
grades. Clearly this is an issue that merits specific attention in future studies. 

To interpret the persistence outcomes, assume that the “true” intervention effect is given 
by the odds ratio for the studies that used a relatively better comparison group (i.e., the odds ratio 
for persistence is 1.62). If we assume — optimistically — that about 50% of students would re- 
enroll the next semester in the absence of the intervention, then the odds ratio suggests that for 
about every 10 students who receive the intervention, one additional student would persist the 
next semester. Of course, due to the fact that the studies included in this review tended not to 
measure outcomes beyond two semesters after the intervention, we know very little about how 
program effects behave over time. 

A study employing the regression-discontinuity design. Moss and Yeaton (2006) used 
regression-discontinuity to study the effectiveness of a developmental English program in a large 
community college. The regression-discontinuity design involves assigning a cut-off point, 
below (or above) which participants receive the intervention. For example, a pretest might be 
administered, and all potential participants falling below a certain threshold score might be given 
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the intervention. This design generally has very strong inferential properties, as, like studies 
using random assignment, the selection mechanism is entirely known. 

In the Moss and Yeaton (2006) study, the college administered a placement exam in 
English. Students falling below a certain threshold score were required to take a developmental 
English course before they could take college-level English. This was the only intervention 
component that was investigated in the present study. The outcome variable was the grade that 
the students earned in their college-level English course. Results suggested that the 
developmental English course was associated with better grades in the college-level course, 
although this effect appears to have been concentrated in the students who scored the lowest on 
the placement test. In other words, students in the developmental course who performed 
relatively well on the placement exam (i.e., those right below the cut-point requiring the course) 
received grades similar to students who fell just on the other side of the cut-point. However, 
students who scored very low on the placement exam received grades in their college-level 
course that were similar to their peers who scored higher on the placement exam. Eollow-up 
analyses suggested that neither differential maturation nor differential attrition, both potential 
rival hypotheses in this particular study, appear to have influenced study results. 

Other Studies Meeting Review Criteria 

We mentioned that we also uncovered studies that met the inclusion criteria but fit into 
other transition paths besides 3 and 4 (i.e., facilitating the transition through college among 
students already in college). We discuss the methods and findings for these in turn. 

Brewer and Landers (2005). Brewer and Eanders (2005) investigated the effects of a 
talent search (TS) program on postsecondary enrollment (Paths 1 and 2 in Eigure 1). These 
programs identify students who demonstrate potential for college study, and this identification 
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usually occurs fairly early (most commonly in middle sehool). This study, conducted at the 
University of Tennessee-Knoxville, offered academic and career advising to program 
participants. TS programs often include additional educational opportunities (e.g., enrichment 
programs). This study notably involved students who were also low-ineome and would be the 
first in their families to graduate from college. 

To study the effects of the TS programs, the authors formed a eomparison group of 
students who were eligible to partieipate in the TS program but for some reason did not. Data 
were eollected between one and nine years after program participation. Results suggested that TS 
program participants were 2.3 times more likely to have enrolled in postsecondary education. 

However, one important limitation of this study is that the comparison group was 
comprised of students who chose not to attend the TS program. One potential rival explanation 
for the finding that TS students were more likely to attend college is that these students differed 
systematically from program participants in a way that might have biased the study findings. In 
faet, TS program participants were 2.6 times more likely than non-participants to have a parent 
who attended at least some college. This suggests that, in fact, the comparison group was 
structured in a way that biased the study results. As such, it is not at all clear whether participants 
in this program were more likely to attend eollege beeause of the Talent Seareh program, 
background factors that made attendance more likely, or a combination of both. 

Kemple (2008). In a long-term randomized experiment, Kemple (2008) investigated the 
effects of career aeademies in eight school districts across the United States (Paths 1, 2, and 7 in 
Figure 1). The sehool districts were selected in part due to the fact that they had relatively mature 
eareer academy programs at the start of the study, were implementing these aeademies in a way 
that conformed with a few critical “best practices” (i.e., utilized relatively smaller learning 
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communities, had an academic curriculum with career-themed courses, and established employer 
partnerships), and made special efforts to recruit students who were perceived to be at high risk 
of dropping out of school. 

Eight years after scheduled graduation, high-risk students who were randomly assigned to 
attend a career academy (a) were somewhat more likely to have earned a high school diploma or 
GED than students who did not attend the academy and (b) were slightly more likely to have 
graduated from high school (odds ratio = 1.07), although this finding was not statistically 
significant. The rates of college attendance between students attending a career academy and 
those not attending a career academy were virtually identical (38.4% vs. 38.6%). The report did 
outline potentially beneficial employment outcomes for career academy students (e.g., somewhat 
higher rates of employment and somewhat greater earnings). 

Maxwell (2001). In a related study. Maxwell (2001) investigated the effects of career 
academies on college achievement and graduation rates (Paths 1 and 2 in Eigure 1). Because her 
interest was in investigating the overall effects of career academies in one particular district. 
Maxwell examined students who enrolled at a university in California from that school district 
and did not provide much detail about the nature of the career academies themselves. She did 
report that the district operated career academies in six different high schools, and it seems 
reasonable to expect differences in the programs across the schools. Maxwell compared career 
academy students to non-academy students by statistically controlling some important 
background variables. Her results suggest that students in career academies need less 
remediation in English than similar non-career academy students and graduate at a slightly 
higher (but still relatively low) rate compared to similar non-academy students; these effects 
appeared to be small. 
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Brancard et al. (2006). Brancard, Baker, and Jensen (2006) evaluated the effeets of a 
eommunity eollege program aimed at students for whom English is a second language (Path 3 in 
Figure 1). The program eompared ESL students involved in a learning eommunity to those who 
were not. Typically these programs are construeted to provide students with soeial support and a 
shared sense of group norms that endorse learning goals. In this study, the learning community 
model integrated language skills (i.e., grammar and composition), featured collaboration among 
faculty, and provided edueational case management for students. Brancard et al. used a quasi- 
experimental design with matching to investigate program effects. Results showed that students 
in the intervention group were about 17% more likely to re-enroll the second semester than were 
students in the comparison group. Further, the intervention group had higher (but not statistieally 
signifieant) eourse eompletion rates and GPAs, but it is unelear from the report how large these 
effeets were. 

Richburg-Hayes et al (2009). Riehburg-Hayes et al. (2009) evaluated the effeets of a 
scholarship program aimed at individuals who had graduated from high school (or earned their 
GED), had a child under age 19, and whose income was less than 200% of the Federal poverty 
level (Paths 1 and 5). Eising a random assignment design, over the eourse of two semesters, 
participants were given supplemental financial aid of $1,000 a semester for (a) enrolling at least 
half time and (b) maintaining at least a C GPA; students also reeeived an enhaneed version of the 
eounseling services available to all students. Results suggested that while the intervention was in 
effect, students in the seholarship group were more likely to enroll in courses relative to 
members of the control group (82% to 77%), were more likely to take a full-time eourse load 
(60% to 54%), and were more likely to maintain at least a C average (55% to 42%). The findings 
were even more impressive for the seeond semester: The seholarship group members were more 
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likely to enroll in courses relative to members of the control group (64% to 49%), were more 
likely to take a full-time course load (46% to 32%), and were more likely to maintain at least a C 
average (38% to 27%). In addition, longer term follow-up results suggested that the relative 
advantage experienced by the scholarship group persisted over time. 

Discussion 

Perhaps the most striking finding from this systematic review is that many interventions 
supporting transition that are of interest to policymakers lack even one experimental evaluation 
and most existing non-experimental evaluations are of undetermined inferential strength. We 
targeted 10 such transitional paths for this review, and only two (Paths 1 and 2 in Figure 1) had 
at least three studies that involved an external comparison group of any kind. In part, this finding 
is a result of our focus on individuals who, from a public policy perspective, could be considered 
disadvantaged. There are some transition programs without this specific focus that have been 
evaluated (e.g.. Tech Prep; Bragg, Loeb, Zamani, & Yoo, 2001). But even within this larger 
group of non-targeted interventions, a high-quality literature base capable of carefully informing 
public policy does not yet exist. 

In addition, the studies we did uncover provide a weak basis for public policy, because 
their designs tend not to be strong; further, they lack reporting on details that would allow us to 
assess the conditions under which and characteristics of students for whom the interventions 
might be effective. As an example, due to poor reporting, we were unable to critically examine 
the quality of the included studies in a rigorous manner. For example, few of the included studies 
discussed implementation fidelity in much depth, and as such was have little information about 
the degree to which observed effects might be attenuated due to low fidelity. Also, most studies 
employed an evaluation design in which students were allowed to choose whether they received 
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the intervention or a comparison condition (e.g., a voluntary course; an analysis of a database 
that tracked student experiences). Although the researchers often took steps to attempt to make 
the intervention and comparison groups more comparable, these designs still carry with them an 
added element of ambiguity. This concern partly exists because we can never know how well our 
attempts to make groups more comparable have worked. As a result of these concerns, we were 
unable to shed additional light on important questions such as the mechanisms by which these 
interventions exert influence (i.e., how they work, assuming they do), which program 
implementation characteristics are associated with better outcomes, and whether transition 
programs seem to be especially effective for students with certain characteristics. Future studies 
using this type of design and analysis strategy should attend more explicitly to the concerns 
raised by non-experimental designs. In addition, a systematic and rigorous program of 
evaluations that are targeted at transition interventions would help clarify whether these are 
effective. 

With respect to interventions that are targeted at college students at risk for dropout, we 
noted that the evidence base we uncovered is not deep, and that the interventions we studied 
varied along a number of critical dimensions. Because of these characteristics, we were unable to 
examine how or why programs might be effective. Even though the data seem to suggest that, 
among our stronger studies, there is evidence that the comprehensive interventions might 
positively affect short-term grades and persistence, we have little information about which 
elements in the comprehensive interventions might be relatively more effective. Future 
evaluations should provide information on the specific elements that were part of the 
intervention strategy and report details about resource utilization. For example, several 
interventions included in our review had a tutoring component, but no studies provided detailed 
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information about the training of tutors or the number of tutoring sessions attended. 

In addition, most studies eontained little information that would help individuals make 
deeisions about how best to support students in partieular areas, sueh as those in career and 
technical education or those in community college settings, and most provided virtually no 
information about program costs. Taken together, these concerns suggest potentially serious gaps 
in our understanding of the effectiveness of specific program elements to support transitions. Our 
hope is that this review spurs rigorous and theoretically rich studies of funded interventions that 
aim to support students as they transition to new roles. 
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APPENDIX 



Table 1 

Postsecondary Transitions Study Screening Guide 



1. First Author (Last, Initial) 




2. Y ear of Publication 




3. Journal 




4. Pages 




Inclusion Criteria 




5. Does this report describe an intervention? 
If No then STOP 


0. No 

1. Yes 

99. Can’t tell, guess = 


6. Is the sample “at-risk” as defined in the relevant 
legislation? 

Note: This is defined broadly, but does not include 
individuals who might be deemed at-risk simply 
due to their race or ethnicity. Common categories 
that do meet the definition are: low SES, under 
prepared or underachieving, disabled. 

If No then STOP 


0. No 

1. Yes 

99. Can’t tell, guess = 


7. Does the intervention address a transition from 
one educational system to another or from an 
educational system to a related career? 

If No then STOP 


0. No 

1 . Yes, one educational system to another 

2. Yes, an educational system to a related 
career 

99. Can’t tell, guess = 



63 





Table 2 

Postsecondary Transitions Study Categorization Guide 



What is the first author’s last name and first initial? 




What was the year of appearance of the report or 
publication? 

What was the type of publication? 

1 = journal article 

2 = book chapter 

3 = book 

4 = dissertation 

5 = MA thesis 

6 = private report 

7 = government report (state or federal) 

8 = school or district report 

9 = other (specify ) 




What kind of transition does the intervention 
address? 

NOTE: If 0 then STOP 


0. No transition 

1 . From one educational system to another 

2. From an educational system to related 
employment 


If the transition is to related employment, briefly 
describe that employment 




Is the intervention aimed at at-risk students? 

Note: Mark “yes” if the intervention is aimed at 
students who are not at risk but this study focuses 
on at-risk students. 

NOTE: If NO then STOP 


0. No 

1. Yes 


Briefly describe the source of the “at-risk” 
designation 
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Note: For example, “Underachieving” 




What is the age range of students in the study? 
Note: Select the “best” answer 


1 . Fligh school students 

2. College students 

3. Multiple age ranges 

4. Can’t tell 

5. Other 


What kind of study is this? 


0. Program description only 

1 . Mainly a program description but with 
some data 

2. Qualitative study 

3. One group pretest-posttest 

4. Comparison group study (i.e., a 
comparison of at least two groups, 
regardless of the kind of comparison) 
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Table 3 

Postsecondary Transitions Study Coding Final Guide (College Persistence Programs) 



What is the first author’s last name and first initial? 




What is the research design? 


0. Non-equivalent groups quasi-experiment 

1 . Randomized experiment 

2. Other 


Are pretest data available for the outcome or a 
closely related proxy? 


0. No 

1. Yes 


Did any participants get to choose their condition? 


0. Y es, all students were volunteers 

1 . Comparison group made up of students 
eligible for program but chose not to attend 

2. No, all students were assigned to their 
conditions (e.g., random assignment study) 

3. Can’t tell 


Does the study appear to have experienced 
attrition? 


0. No 

1. Yes 

2. Can’t tell 


If yes, list attrition rates if available. 


Overall 

Differential 


Do any study participants appear to have switched 
groups during the study? (e.g., moved from 
treatment to control? 


0. No 

1. Yes 

2. Can’t tell 


Outcomes 


This is outcome of 


What construct is this outcome tapping? 


1 . Academic achievement 

2. Persistence 

3. Other 


What is the source for the data? 


0. Self-report 

1. Teacher report 

2. Archival records (includes grades) 

3. Other 


What is the timing of the outcome assessment? 


0. Immediately at the end of the intervention 

1 . At the end of the semester in which the 
intervention was delivered 

2. One frill semester after the intervention was 
delivered 

3. More than one frill semester after the 
intervention was delivered 
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What intervention components were included? (list 
all) 


0. Study skills course/seminar 

1 . Adjusting to college life course/seminar 

2. Tutoring 

3. Mentoring 

4. Differential policies (e.g., limitations on 
the number of courses) 

5. Other 


If the intervention included a course or seminar, 
how long was it? 


weeks 


Did the report give an indication of resource 
utilization? (Example: average number of tutoring 
sessions attended.) 


0. No 

1. Yes 


Average sample age 


years 


Range of sample age 


to years 


Ethnicities represented in sample (circle all that 
apply) 


0. White 

1 . African-American 

2. Eatino 

3. Asian- American 

4. American Indian 

5. Other 

6. Not specified 


SES represented in sample 


0. Eow SES 

1 . Eower-middle SES 

2. Middle SES 

3. Middle-upper SES 

4. Mixed 

5. Not specified 


Were students in a community college or in a four- 
year institution? 


0. College/University 

1 . Community college 

2. Can’t tell 


Effect Size Data 




To whom were program students compared? 


0. Students not meeting inclusion criteria 
(e.g., typical college students) 

1 . Students meeting inclusion criteria 

2. Can’t tell 


Were adjusted means available in the report? 


0. No 

1. Yes 


If yes, for what variables were the means adjusted? 


0. n/a, no adjusted means 

1. Prior achievement (e.g., high school gpa) 

2. Standardized achievement test (e.g., SAT 
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Table 4 

Studies Measuring Academic Achievement Outcomes 



First Author 
(Year) 


Intervention Description 


Target Population/ 
Setting 


Duration of 
Intervention 


Comparison Group 


Outcome 


Outcome Assessment Timing 


Effect 

Size“ 


Abadie(1999) 


Administrative limitations 
on extracurricular 
activities, smaller class 
sizes, required body of 
general education courses 


Incoming 4-year 
college students not 
meeting regular 
admit criteria 


One academic 
year 


Students admitted 
via usual admission 
process 


GPA 


After first semester of 
intervention 

After second semester of 
intervention 


-0.13 

-0.56 


Alderman 

(1998) 


One credit college 
orientation class, tutoring, 
remedial coursework 


Community college 
students identified as 
needing remedial 
instruction 


One semester 


Historical controls 
who met inclusion 
criteria (i.e., students 
before program 
existed) 


GPA 


Semester following the end of 
the intervention 


+.18 


Clark (1993) 


Remedial coursework, 
small classes, academic 
and career advising 


Incoming 4-year 
students scoring in 
the lowest quartile of 
a placement test 


One academic 
year 


Historical controls 
who met inclusion 
criteria (i.e., students 
before program 
existed) 


GPA 


Three and a half years after the 
end of the intervention 


+0.93 


Cohen 

Goodman 

(1998) 


Added a journal writing 
component to an English 
composition class 


Students in a 4 year 
university scoring 
low on a placement 
test 


One semester 


Randomly group of 
students not assigned 
to intervention 
condition 


Reading 

comprehension 


Immediately after intervention 


+0.07 


Cone (1991) 


Study skills and 
adjustment course 


Students in a 4 year 
university with first 
semester gpa < 2.0 


One semester 


Unclear, but 
comparison students 
do not appear to 
have met inclusion 
criteria for 
intervention 


GPA 


Immediately after intervention 


-0.61 


Cox (2002) 


Study skills curriculum 
integrated into usual math 
instruction 


Students in a 
community college 
scoring below cutoff 
on a placement test 


One semester 


Historical controls 
who met inclusion 
criteria (i.e., students 
before program 
existed) 


Grade in one 
math class 


Immediately after the 
intervention 


+0.32 


Dees (1991) 


Cooperative learning in a 
remedial math class 


Students in a 4 year 
university needing 


One semester 


Randomly assigned 
group of students in 


Various math 
tests 


Immediately after the 
intervention 


+0.37 
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First Author 
(Year) 


Intervention Description 


Target Population/ 
Setting 


Duration of 
Intervention 


Comparison Group 


Outcome 


Outcome Assessment Timing 


Effect 

Size“ 






remediation 




a traditional lab 








Esterbrook 

(2006) 


Behavior modification 


Students in a 
community college 
subjectively deemed 
to be “at-risk” 


Unclear, but 
appears to be 
one semester 


Assigned to receive 
traditional remedial 
instruction 


GPA 


Immediately after the 
intervention 


-0.30 


Fry (2007) 


Course aimed at fostering 
time management and 
problem solving skills, as 
well as increased 
awareness of university 
resources 


Conditionally 
admitted students in 
a 4 year university 


One semester 


Conditionally 
admitted students 
not taking the 
seminar 

Unconditionally 
admitted students 
not taking the 
seminar 


GPA 


Unclear, but appears to be for 
the semester during which the 
seminar took place 


+0.21 

-0.54 


Hecker(1995) 


Administrative limitations 
on maximum credit hours, 
courses available, class 
sizes; seminar to teach 
academic skills 


Students 
conditionally 
admitted to a 4 year 
university through 
an alternate process 


Probably one 
academic year 


Regularly admitted 
students deemed to 
be “high risk” 

Regular admits 


GPA 


Immediately following the end 
of the intervention (i.e.. Fall 
semester of the Sophomore 
year) 


+0.15 

-0.58 


Loiacano 

(2000) 


Specific curriculum added 
to an existing freshman 
orientation course 


First year students in 
a 4 year university 
with a history of 
academic struggles 


One academic 
semester 


Similar students not 
receiving the added 
curriculum 


Cognitive 

development 


Immediately after the end of 
the intervention 


+0.11 


McGee (2004) 


Statewide program 
providing financial, 
academic, and social 
assistance 


Disadvantaged 
students attending 
community college 


Unclear, but 
presumably 2 
years 


Students not 
participating in the 
program 


GPA 


Unclear 


-0.22 


McGregor 

(2001) 


Added component to an 
existing college 
preparation course 


Entering freshmen 
not meeting usual 
admission criteria to 
a 4 year university 


5 weeks 


Similar students not 
receiving the added 
component 


Vocabulary 

Critical 

thinking 


Immediately after the end of 
the intervention 
Immediately after the end of 
the intervention 


+0.16 

+0.16 


Milligan 

(2007) 


Study skills seminar 


Students on 
academic probation 
in a 4 year university 


8 weeks 


Similar students who 
chose not to 
participate in the 
seminar 


GPA 


Immediately after the end of 
the intervention 


+0.22 


Salinitri 

(2005) 


Mentoring 


Students with 
entrance scores near 


Unclear, but 
probably one 


Similar students not 
chosen to receive 


GPA 


Unclear, but probably refers to 
the semester immediately 


+0.60 
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First Author 
(Year) 


Intervention Description 


Target Population/ 
Setting 


Duration of 
Intervention 


Comparison Group 


Outcome 


Outcome Assessment Timing 


Effect 

Size“ 






the institution’s 
lower limit 


semester 


mentoring 




following the intervention 




Sanders 

(2000) 


Peer tutoring 


Academically 
underprepared 
freshmen in a 4 year 
institution 


One academic 
year 


Unclear, but seems 
to be similar 
students who did not 
receive the 
intervention 


GPA 


Immediately after the end of 
the intervention 


+0.61 


Scrivner 

(2008) 


Learning communities of 
about 25 students; each 
community took a set of 
three courses together; 
curricula across the 
courses were linked. 
Tutoring was also offered. 


Freshmen at a 
community college 
(sample in the meta- 
analysis had failed 
both English 
placement tests 
given by the college) 


One semester 


Students randomly 
assigned to receive 
college’s usual menu 
of courses and 
support 


Whether 
students had 
passed an 
English 
course'’ 


First full semester after the end 
of the intervention 


+.22 


Stovall (1999)“ 


Student success course 
focusing on transitioning 
to college, career 
development, and life 
management. 


Students in a 
community college 


One semester 


Students who scored 
below college level 
on two placement 
tests (reading and 
English) 


GPA 


Immediately after the end of 
the intervention‘s 


+0.21 



Note. “All effect sizes for academic achievement are expressed as standardized mean differences. A standardized mean difference > 0 indicates that the students 
receiving the intervention performed better than students in the comparison condition 
In Scrivner et al. (2008), this outcome was expressed in terms of percentages of students who had passed an English course vs. those who had not. We 
computed a logged odds ratio for this outcome, then transformed that logged odds ratio to a standardized mean difference effect size. 

“ Stovall (1999) did not analyze students who took but did not pass the student success course. This choice may positively bias the effect size estimate somewhat. 

Stovall (1999) also measured GPA at the end of the second term, second academic year, and third academic year. She found no differences between program 
participants and non-participants and did not separately compare at-risk program participants to at-risk non-participants. For meta-analysis, we conservatively 
imputed 0 for these effects. 
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Table 5 

Studies Measuring Persistence Outcomes 



First Author 
(Year) 


Intervention Description 


Target Population/ 
Setting 


Duration of 
Intervention 


Comparison Group 


Outcome 


Outcome Assessment Timing 


Effect 

Size“ 


Abadie (1999) 


Administrative limitations 
on extraeurrieular 
aetivities, smaller elass 
sizes, required body of 
general edueation eourses 


Ineoming 4-year 
eollege students not 
meeting regular 
admit eriteria 


One academic 
year 


Regular admits 


Retention 


First semester after 
intervention end 

Seeond semester after 
intervention end 


.36 

.42 


Alderman (1998) 


One eredit eollege 
orientation elass, tutoring, 
remedial eoursework 


Community eollege 
students identified as 
needing remedial 
instruetion 


One semester 


Historieal eontrols 
who met inelusion 
eriteria (i.e., students 
before program 
existed) 


Retention 


First semester after 
intervention end 


1.32 


Clark (1993) 


Remedial eoursework, 
small elasses, aeademie 
and eareer advising 


Ineoming 4-year 
students seoring in 
the lowest quartile of 
a plaeement test 


One academic 
year 


Historieal eontrols 
who met inelusion 
eriteria (i.e., students 
before program 
existed) 


Retention 


Three and a half years after the 
end of the intervention 


1.21 










Regular admits 






.86 


Cone (1991) 


Study skills and 
adjustment eourse 


Students in a 4 year 
university with first 
semester gpa < 2.0 


One semester 


Historieal eontrols 
who met inelusion 
eriteria (i.e., students 
before program 
existed) 


Retention 


Semester following the 
intervention 


14.48 


Fry (2007) 


Course aimed at fostering 
time management and 
problem solving skills, as 
well as inereased 
awareness of university 
resourees 


Conditionally 
admitted students in 
a 4 year university 


One semester 


Conditionally 
admitted students 
not taking the 
seminar 

Uneonditionally 
admitted students 
not taking the 


Retention 


Semester following the 
intervention 

Two semesters following the 
intervention 

Semester following the 
intervention 


1.12 

.88 

.76 



72 





First Author 
(Year) 


Intervention Description 


Target Population/ 
Setting 


Duration of 
Intervention 


Comparison Group 


Outcome 


Outcome Assessment Timing 


Effect 

Size“ 










seminar 




Two semesters following the 
intervention 


.64 


Hecker(1995) 


Administrative limitations 
on maximum credit hours, 
courses available, class 
sizes; seminar to teach 
academic skills 


Students 
conditionally 
admitted to a 4 year 
university through 
an alternate process 


Probably one 
academic year 


Regularly admitted 
students deemed to 
be “high risk” 

Regular admits 


Retention 

Retention 


Immediately following the end 
of the intervention (i.e.. Fall 
semester of the Sophomore 
year) 


.77 

.49 


House (1991) 


Tutoring 


Academically under 
prepared freshmen 
admitted through a 
special process to a 4 
year university 


Probably one 
academic year 


Students eligible to 
receive tutoring but 
who did not 


Retention 


Appears to be the next 
semester after the end of the 
intervention (i.e.. Fall semester 
of the Sophomore year) 


1.52 


Milligan (2007) 


Study skills seminar 


Students on 
academic probation 
in a 4 year university 


8 weeks 


Similar students who 
chose not to 
participate in the 
seminar 


Retention 


Immediately after the end of 
the intervention 


1.15 


Salinitri (2005) 


Mentoring 


Students with 
entrance scores near 
the institution’s 
lower limit 


Unclear, but 
probably one 
semester 


Similar students not 
chosen to receive 
mentoring 


Retention 


Unclear, but probably refers to 
the semester immediately 
following the intervention 


14.60 


Sanders (2000) 


Peer tutoring 


Academically 
underprepared 
freshmen in a 4 year 
institution 


One academic 
year 


Unclear, but seems 
to be similar 
students who did not 
receive the 
intervention 


Retention 


Immediately after the end of 
the intervention 


.83 


Stovall (1999)*’ 


Student success course 
focusing on transitioning 
to college, career 
development, and life 
management. 


Students in a 
community college 


One semester 


Students who scored 
below college level 
on two placement 
tests (reading and 
English) 


Retention 


Immediately after the end of 
the intervention 

End of the second academic 
year 


23.69 

1.94 














End of the third academic year 


1.39 



Note. ’“All retention effect sizes are expressed as odds ratios. An odds ratio > 1 indicates that the intervention was associated with increased retention. 

Stovall (1999) did not analyze students who took but did not pass the student success course. This choice may positively bias the effect size estimate somewhat. 



73 







CTE 



National Research 
Center for Career and 
Technical Education 



National Research Center for Career and Technical Education 

University of Louisville, College of Education and Human Development, Louisville, KY 40292 
Phone: 502-852-4727 • Toll-Free: 877-372-2283 • Fax: 502-852-3308 
Email: nrccte@louisville.edu • www.nrccte.org 



